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ABSTRACT:  Predicting  very  high  or  low  EED  response 
probabilities  usually  requires  extrapolation  from  limited 
data.  Extrapolation  errors  will  be  increased  by  inappropriate 
test  plans  or  statistical  treatment.  Various  test  plans 
are  discussed.  Possible  new  prediction  techniques  are 
suggested.  The  necessity  for  careful  design  of  experiment 
is  stressed. 
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Estimation  of  High  and  Low  Probability  EED  Functioning  Levels 


The  present  report,  with  minor  variations,  was  presented 
at  the  4th  Electric  Initiator  Symposium,  1  &  2  October  1963. 
Because  the  subject  material  is  of  broad  interest  it  warrants 
a  wider  distribution  than  it  might  get  by  appearing  only  in 
the  Proceedings  of  the  Symposium.  The  concepts  dealt  with 
are  equally  of  interest  in  weapon  safety  and  weapon  reliability 
considerations . 

The  work  leading  to  the  writing  of  this  report  was  supported 
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I.  INTRODUCTION 

In  this  paper  the  authors  wish  to  address  themselves  to  the 
problem  where,  with  a  limited  number  of  samples,  it  is  desired 
to  predict  the  stimulus  corresponding  to  an  extreme  functioning 
probability  level  for  a  given  electric  initiator  population,  or 
conversely  the  estimated  response  at  a  stipulated  stimulus.  This 
problem  is  becoming  increasingly  important  to  both  the  military 
and  the  space  agencies.  In  the  past  the  military  could  frequently 
tolerate  weapons  having  a  relatively  large  degree  of  unreliability 
and  then  compensate  for  this  unreliability  by  firing  large  numbers 
of  weapons  to  attain  the  desired  target  kill;  for  example,  the 
firing  of  projectiles  or  the  dropping  of  bombs  in  large  quantities. 
However,  complex  modern  weapons,  their  high  cost  and  their  great 
destructive  power  often  preclude  firings  in  large  nuitfeers.  High 
reliability  (and  safety)  must  be  achieved  and  demonstrated  for 
the  individual  weapon.  As  for  space  ventures  the  complexity  of 
operations,  the  necessity  for  accuracy,  the  high  cost,  the  prestige 
value,  and  the  stake  in  human  lives  make  it  mandatory  that 
components  have  a  high  level  of  reliability  and  safety. 

High  reliability  (or  safety)  in  the  sense  that  we  will  use 
it  here  is  a  functioning  probability  of  99.5%  or  higher  at  a 
specified  input  level.  Such  reliabilties  are  not  excessive  for 
electro-explosive  devices.  Experience  based  on  thousands  of 
manufacturers'  firings  of  conventional  primers  and  detonators 
show  that  such  reliabilities  are  in  fact  usually  exceeded  by 
ordinary  production  techniques.  During  the  course  of  development, 
however,  it  is  often  necessary  to  predict  the  response  of  EED's 
to  given  stimuli.  Fer  example,  in  assessing  hazards  of  electro¬ 
magnetic  radiation  it  may  be  necessary  to  predict  the  response  at 
a  very  low  stimulus  level.  To  determine  whether  a  given  power 
supply  in  a  particular  weapon  is  capable  of  reliably  firing  an 
EED  it  is  necessary  to  estimate  the  response  of  the  EED  to  the 
input  stimulus  of  the  power  supply. 

The  direct  demonstration  of  a  99.5%  or  better  response  at 
95%  confidence  of  an  EED  to  a  given  stimulus  is  often  too  costly 
in  material,  time,  and  manpower  to  be  seriously  considered.  It 
would  require  the  firing  of  approximately  750  items  without  a 
failure . 

Before  discussing  the  general  philosophy  for  making, logically> 
the  required  estimates,  some  discussion  appears  warranted  about 
the  present  most  frequently  used  method.  This  is  the  Bruceton 
method. 

It  is  the  authors'  observation  that  the  Bruceton  test 
method  is  being  used  extensively  for  determining  the  response 
(sensitivity)  of  electric  initiators.  When  properly  used  it  is 
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a  good  method.  It  is  rapid  and  economical.  Tha  algebraic 
manipulations  required  to  preduce  the  statistical  quantities 
are  simple  to  carry  out.  It  is  because  of  these  features  that 
the  Bruceton  test  has  found  such  widespread  application. 
Unfortunately  it  has  bean  frequently  used  in  situations  where 
the  results  obtained  are  inaccurate  and  misleading. 

For  making  studies  areund  the  50%  response  level  the  test 
is  mest  often  highly  acceptable  and  advantageous.  When 
estimates  are  made  by  the  Bruceton  method  beyond  the  75%  response 
level  difficulties  can  be  anticipated.  The  authors  have  spelled 
out  in  detail  the  reasons  for  the  difficulty  in  a  paper 
presented  before  the  last  HERO  Cong *«ssl*.  The  salient  reasons, 
without  detail,  are  worthy  of  repetition: 

a.  The  Bruceton  method  gives  a  very  poor  estimate 
of  the  standard  deviation.  Even  Bruceton  tests 
of  100  samples  will  often  underestimate  the 
true  standard  deviation  by  50  per  cent  or  more. 

b.  Since  most  all  of  the  data  are  collected  between  the 
25  and  75%  firing  points,  long  extrapolations 

must  be  made  to  the  points  of  interest  along 
a  curve  which  is  usually  unknown. 

When  it  is  not  feasible  to  demonstrate  directly  a  response 
at  an  extreme  firing  point,  estimates  of  the  response  are  usually 
made  by  a  process  of  extrapolation  and  curve  fitting.  The 
extrapolation  process  is  basic  to  the  approach.  This  principle 
should  be  kept  firmly  in  mind.  All  of  us  as  technical  people 
are  very  familiar  with  making  extrapolations  and  the  principles 
involved.  The  statistical  problems  are  really  no  different.  What 
is  desired  is  an  extrapolation  from  measured  response  points  to 
points  removed  from  the  region  of  measurement.  Our  extrapolations 
become  better  as  the  length  of  the  extrapolation  becomes  smaller. 
They  also  become  better  when  the  general  shape  of  the  curve  being 
extrapolated  is  known.  From  the  statistical  standpoint,  when  the 
response  function  or  distribution  function  is  known  in  the  region 
of  extrapolation. 

There  is  no  single  best  method  for  making  estimates  of 
extreme  functioning  probability  points.  Various  methods  are 
available  for  use.  Those  which  can  be  used  for  best  results 
depend  on  such  factors  as  sample  size  available,  the  degree  of 
accuracy  needed,  data  available  from  other  tests,  and  the 
remoteness  of  the  desired  functioning  level. 


♦References  are  given  on  page  iii. 
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II.  PRESENT  SENSITIVITY  TESTING  METHODS 

Sensitivity  tests  are  of  different  types.  Each  type  has 
certain  advantages  and  disadvantages.  These  should  be  considered 
to  make  an  intelligent  selection  of  the  test  to  be  used.  In 
certain  situations  one  test  would  be  selected  while  in  others  a 
different  test  would  be  chosen.  We  shall  consider  some  of  the 
tests  which  are  frequently  used  along  with  their  advantages  and 
disadvantages.  First,  however,  it  would  be  wise  to  state  some 
principles  which  will  be  general  in  their  applications . 

In  most  tests  the  analysis  involves  fitting  a  frequency 
distribution  function  to  the  observed  data.  In  other  words  the 
test  consists  of  an  experiment  in  which  the  sensitivity  is 
determined  at  each  of  two  or  more  stimulus  levels.  From  these 
data  we  attempt  to  predict  either  the  response  at  some  other  level 
or  the  level  which  will  have  some  desired  response.  In  order  to 
do  this  we  must  assume  some  frequency  distribution  function. 

One  which  has  been  widely  used  in  the  explosives  field  is  the 
log-normal  function.  Experience  has  shown  that  this  is  a  fairly 
good  fit  and  entirely  adequate  for  many  purposes.  However, 
recent  work  at  The  Franklin  Institute*  and  at  the  Naval  Ordnance 
Laboratory*  has  shown  that  the  log-logistic  function  gives  a 
somewhat  better  fit.  Even  this  is  not  a  perfect  fit. 

In  general,  predictions  based  upon  interpolation  from 
observed  data  are  fairly  safe  since  the  function  assumed  in 
the  interpolation  will  ordinarily  coincide  closely  with  the 
true  function  over  the  range  of  the  observed  data.  On  the 
other  hand  the  assumed  and  true  functions  may  differ  considerably 
outside  this  range.  For  this  reason  extrapolation  is  always 
dangerous  because  of  the  uncertainty  in  the  choice  of  distribution 
function.  The  larger  the  extrapolation  the  greater  the  resulting 
error  is  likely  to  be .  The  use  of  extrapolation  cannot  be  avoided 
in  estimates  of  very  high  or  low  response  points .  However  it 
can  be  kept  small  by  proper  choice  of  test  plan  at  the  cost  of 
testing  an  increased  number  of  items. 

A  second  point  to  be  considered  is  the  possibility  of  bias. 
Some  tests  have  a  tendency  to  over  or  underestimate  the  quantity 
which  is  being  determined.  This  tendency  is  known  as  bias. 

Some  bias  might  be  tolerated, if  it  were  in  the  direction  of 
making  a  more  conservative  estimate. 

Another  point  to  be  considered  in  planning  sensitivity 
experiments  is  the  allocation  of  items  to  the  stimulus  test 
levels.  A  trial  made  at  a  stimulus  level  at  which  almost  all 
trials  are  expected  to  result  in  fires  or  fails  gives  us  less 
information  than  one  made  near  the  fifty  per  cent  point.  To 
obtain  an  equal  amount  of  information  at  each  level  we  must 
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assign  larger  numbers  of  items  at  levels  farther  from  the  fifty 
per  cent  point.  By  this  method  we  can  give  each  of  the  levels 
equal  weight. 

Another  consideration  is  the  total  number  of  items  to  be 
tested.  Of  course, the  larger  this  number  the  more  information 
we  obtain.  This,  then,  usually  becomes  a  compromise  between  the 
amount  of  information  we  would  like  to  have  and  what  we  can 
afford  to  spend  in  time  and  money  in  order  to  get  it.  8ome  tests 
are  more  efficient  than  others  in  obtaining  information  from  a 
given  number  of  trials. 

One  type  of  test  which  is  quite  largely  used  is  the  up-and- 
down  or  stair-step  test,  the  best  known  being  the  Bruceton  test. 

This  test  concentrates  the  trials  near  the  fifty  per  cent  point. 

All,  or  nearly  all,  of  the  data  will  be  from  observations  concentrated 
between  the  25  and  75  per  cent  points.  The  weights  of  the  observa¬ 
tions  at  the  test  levels  will  show  an  even  greater  concentration 
around  the  fifty  per  cent  point.  Investigations  in  England  and 
at  the  Naval  Ordnance  Laboratory  have  indicated  that  the  Bruceton 
test  has  a  serious  bias  in  the  estimation  of  the  standard 
deviation, giving  a  value  which  is  too  small.  The  effect  of  this 
bias  would  be  to  predict  too  much  reliability  and  safety  for  an 
item  which  is  tested  in  this  way.  The  error  becomes  even  more 
serious  since  the  concentration  of  trials  near  the  fifty  per  cent 
point  makes  the  prediction  of  reliability  or  safety  depend  upon 
extreme  extrapolation.  Consideration  of  the  characteristics  of 
the  Bruceton  test  shows  that  it  is  a  good  test  for  anyone  who  is 
interested  in  determining  the  fifty  per  cent  point  but  a  poor 
test  for  determining  high  or  low  per  cent  points. 

Another  test  which  has  some  of  the  characteristics  of  an 
up-and-down  test  is  the  Bartlett  test.  Stimulus  levels  are  set 
up  and  testing  continued  at  each  level  until  two  reversals  are 
observed.  A  reversal  is  a  fire,  or  fail,  when  the  other  response 
is  expected.  The  Bartlett  plan  gives  an  increasing  number  of 
trials  as  we  get  farther  from  the  fifty  per  cent  point.  Thus 
the  weights  of  the  observations  at  the  different  levels  are  made 
approximately  equal.  It  also  reduces  the  extrapolation  required 
for  very  high  or  low  response  points  and  therefore  is  a  good  test 
for  making  estimates  of  extreme  functioning  levels.  It  is  fairly 
easy  to  show,  however,  that  estimates  of  sensitivity  obtained  by 
this  plan  are  biased.  Sixty  per  cent  of  the  tests  will  give 
estimates  of  the  sensitivity  which  are  too  low  at  the  upper  end  of 
the  range  and  too  high  in  the  lower  end.  This  bias  is  not  as 
serious  as  that  shown  by  the  Bruceton  test  since  it  is  in  the 
direction  of  conservatism.  It  should  be  emphasized  that  the 
Bartlett  test  requires  very  large  samples.  In  two  instances** * 
the  sample  sizes  were  approximately  8000. 
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A  third  type  of  teat  is  one  which  has  been  analyzed  by 
Golub  and  Grubbs4  of  the  Ballistic  Research  Laboratory  at 
Aberdeen,  Maryland.  In  this  type  of  test  a  comparatively  small 
number  of  items  is  tested  at  different  levels  of  stimuli  with 
possibly  only  one  item  at  each  level.  This  type  of  plan  is 
especially  applicable  to  tests  in  which  the  stimulus  level  cannot 
be  exactly  controlled  but  can  be  measured.  Ford  Motor  Company* 
has  recently  done  some  work  on  a  similar  type  of  test.  Since  the 
sample  size  for  either  of  these  tests  is  usually  small,  the 
results  are  subject  to  the  uncertainty  always  associated  with 
small  samples. 

Finally,  a  plan  which  is  quite  frequently  used  is  one  that 
has  been  called  the  run-down  test.  This  type  and  the  up-and-down 
tests  include  most  of  the  sensitivity  tests  which  are  made.  The 
plan  calls  for  making  a  specified  number  of  trials  at  each  of  two 
or  more  stimulus  levels.  We  shall  describe  here  ir  detail,  as  an 
example  of  tailoring  tests  to  specific  situations,  a  run-down  test 
plan  which  calls  for  testing  at  two  stimulus  levels.  This  plan 
was  devised  tc  determine  high  probability  of  firing  estimates  for 
electro-explosive  devices  of  one  of  the  Navy's  most  important 
missiles.  Only  200  EED's  per  sample  were  available  for  test. 

This  test  plan  was  optimized  to  fit  the  specific  needs  but  may 
be  useful  to  others  faced  with  a  similar  problem.  The 
probability  points  of  interest  are  estimated  by  extrapolation 
based  on  observed  responses  measured  in  the  neighborhood  of  the 
65  and  90  per  cent  points.  If  we  have  previous  experience  with 
similar  items  we  may  use  this  experience  as  the  basis  for  choosing 
these  two  test  levels.  Lacking  this  experience, we  can  use  a 
short  Bruceton  test.  Suppose  that  we  use  twenty  of  the  two 
hundred  items  in  the  preliminary  Bruceton  test.  Then  the  remain¬ 
ing  one  hundred  eighty  are  used  for  the  main  test  at  the  two 
levels.  Fifty  items  will  be  allocated  at  the  expected  65  per 
cent  level,  and  the  remaining  one  hundred  thirty  will  be  tested 
at  the  expected  90  per  cent  level.  If,  after  testing  the  fifty 
items  at  the  first  level,  it  appears  that  the  response  is  much 
higher  than  the  expected  65  per  cent  point  we  can  revise  our 
plan  by  using  this  as  the  second  level  rather  than  the  first.  In 
this  case  a  new  level  is  selected  as  the  first, somewhat  nearer  the 
50  per  cent  point,  and  fifty  items  tested  at  the  new  level.  The 
following  is  a  step-by-step  procedure  for  firing  the  two  hundred 
items . 

a.  Fire  twenty  items  in  a  Bruceton  test  to  obtain 
preliminary  estimates  of  the  mean,  m,  and  the 
standard  deviation,  s.  A  log-transform  of  the 
dosage  (current,  potential,  energy)  is  taken 
as  the  stimulus . 


5 

UNCLASSIFIED 


UNCLASSIFIED 
NOLTR  63-266 


b.  Compute  the  first  and  second  test  levels  as 
the  mean  of  the  Bruceton  test  plus  0.4s  and 
1.3s  respectively. 

c.  Test  fifty  items  at  the  first  stimulus  level. 

(1)  If  five  or  fewer  fails  are  observed, 
redefine  the  first  level  as  the  second 
and  continue  firing  ?.t  this  level  until 
one  hundred  thirty  are  tested.  Test  the 
remaining  fifty  at  a  stimulus  level  m  +  0.2s. 

(2)  If  more  than  five  fails  are  observed  (the 
usual  case)  test  the  remaining  one  hundred 
thirty  units  at  the  original  second  level. 

The  analysis  of  the  data  obtained  from  a  test  of  this  kind 
would  require  fitting  a  frequency  distribution  function, as  was 
pointed  out  earlier  in  this  paper.  As  was  also  pointed  out  the 
log-logistic  function  is  the  preferable  one.  The  procedure  for 
fitting  this  function  to  these  data  would  be  as  follows.  First, 
convert  the  observed  number  of  fires,  x,  and  fails,  y,  for  each 
level  into  logits  by  the  relation 

L  -  In  . 

Plot  these  values  of  L  against  the  stimulus  (log-current,  log- 
potential,  or  log-energy) .  Draw  a  straight  line  through  these 
two  points.  To  interpret  the  graph  in  terms  of  per  cent  response 
for  any  stimulus  read  the  result  in  logits  and  change  to  per 
cent  by  the  relation 


where  p  is  the  desired  per  cent. 

A  test  of  this  general  type  has  the  good  feature  of 
minimizing  the  necessary  extrapolation.  It  is  free  from  bias 
such  as  is  found  in  the  Bruceton  or  Bartlett  tests.  The  items 
are  allocated  to  the  test  levels  so  as  to  give  nearly  equal 
weight  to  the  observations.  Two  hundred  items  is  about  as  small 
a  number  as  can  be  used  in  order  to  give  a  good  estimate  of  a 
high  or  low  per  cent  point. 

III.  NEW  APPROACHES 

NOL  is  looking  for  ways  for  improving  extreme-probability 
estimation  methods  by  using  information  in  addition  to  Go/No-Go 
firing  data. 
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As  has  been  pointed  out,  the  estimation  of  very  high  or 
low  probabilities  on  the  basis  of  Go/No-Go  data  always  requires 
extrapolation  towards  the  asymptotic  All-Fire  and  No-Fire  limits. 

The  extrapolation  is  a  risky  business.  Can  we  avoid  this 
extrapolation?  We  think  that  it  is  possible.  By  using  data  from 
such  sources  as  nondestructive  measurements  of  BED  thermal  parameters 
in  conjunction  with  sensitivity  data  we  can  interpolate  rather 
than  extrapolate. 

For  instance,  we  can  show  with  the  Mk  1  Squib  that  a 
current  of  50  milliamperes  through  the  bridge  would  cause  a 
maximum  elevation  of  the  bridgewire  temperature  above  ambient 
of  about  10* Centigrade .  By  figuring  backwards  from  a  maximum 
acceptable  elevation  of  the  bridgewire  temperature,  we  can  deduce 
an  even  higher  maximum  current  which  would  be  acceptable  not  only 
as  a  safe  current  but  also  one  which  will  not  deteriorate  the 
EED.  In  this  fashion  we  can  establish  a  true  No-Fire  current 
level. 

Once  a  non-zero  No-Fire  level  is  available  we  should  be 
able  to  estimate  a  very  low  probability  of  firing  by  interpolation 
between  the  No-Fire  level  and  experimentally  observed  low- 
probability  firing  data. 

A  similar  use  of  the  electro-thermal  data  in  conjunction  with 
limits  of  variability  of  EED  configuration  and  explosive  ignition 
temperatures  should  permit  the  computation  of  a  finite  All-Fire 
point  (provided  there  are  no  Q-C  defects) .  With  this  All-Fire 
point  and  appropriate  firing  data  we  should  be  able  to  interpolate 
to  find  a  high  reliability  point. 

4 

In  either  case,  the  interpolation  can  be  carried  out  only 
,  if  some  distribution  function  can  be  assumed  to  connect  the  data, 

i  There  are  many  expressions  which  can  be  devised  to  describe  a 

f  distribution  which  is  (1)  approaching  zero  probability  tangentially 

at  a  non-zero  positive  No-Fire  point,  (2)  approaching  a  probability 
of  1  at  a  finite  All-Fire  point,  and  (3)  a  fit  through  observed 
firing  data.  What  basis  do  we  have  for  selecting  the  proper 
function? 

To  handle  this  problem,  we  are  investigating  the  field  of 
non-parametric  or  distribution-free  statistics.  The  general 
approach  in  this  technique  is  to  find  facts  which  apply  to 
whole  classes  of  distributions.  It  is  assumed,  on  the  basis  of 
experience,  that  the  EED  distribution,  though  unknown,  falls  in 
a  general  class.  If  appropriate  boundaries  or  limits  for  the 
class  of  distributions  can  be  found  then  it  will  be  possible  to 
set  conservative  bounds  on  the  EED  probability  estimates. 
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For  instance,  if  it  can  be  assumed  that: 

(a)  The  Probability  Density  Function  is  unimodal, 
i.e.,  the  Cumulative  Distribution  Function 
(C.D.F.)  has  a  single  inflection  point  which 
corresponds  to  the  mode  above,  and 

(b)  The  two  distribution  functions  are  zero  at 
the  true  No-Fire  level , 

then  we  can  say  that  a  straight  line  drawn  on  the  C.D.F.  from 
the  No-Fire  point  to  the  inflection  point  will  always  be  more 
conservative  for  safety  estimates  than  any  distribution  which 
satisfies  the  above  criteria.  This  is  because  the  C.D.F.  will 
always  be  concave  upward  in  this  range. 

The  trouble  with  the  above  example  is  the  difficulty  in 
experimentally  locating  the  point  of  inflection  of  the  C.D.F. 

IV.  CONCLUSIONS 

We  advise  caution  and  forethought  in  carrying  out 
sensitivity  determinations.  Ready  made  test  plans  (such  as  those 
previously  mentioned)  have  been  devised  to  answer  specific  needs 
and  have  been  based  on  assumptions  which  are  often  implicit. 

If  these  needs  and  assumptions  are  not  relevant  to  the  current 
problem,  trouble  can  arise.  A  cookbook  firing  plan,  applied 
blindly,  can  be  a  waste  of  time,  money,  and  material. 

The  experimental  and  computational  procedures  should 
therefore  be  carefully  designed  before  the  investigation  is 
started.  The  questions  to  be  answered  should  be  clearly  stated. 
All  relevant  background  and  previous  knowledge  should  be 
considered.  After  an  experimental  program  has  keen  proposed,  the 
interpretations  of  all  foreseeable  sets  of  results  should  be 
hypothesized  before  any  firing  is  commenced.  If  the  possible 
or  likely  outcomes  are  inconclusive,  then  the  experimental 
program  should  be  modified  appropriately.  The  aid  of  a 
statistician  throughout  this  planning  stage  is  very  necessary. 

It  will  reduce  the  probability  of  obtaining  useless,  errant  or 
meaningless  results. 

And  for  results  to  be  useful  to  those  other  than  the 
experimenter,  the  background  information,  the  assumptions  and 
statistical  procedures  should  be  a  part  of  the  data.  They 
should  be  given  in  enough  detail  to  permit  reconstruction  of 
the  logic  used  throughout  the  investigation. 
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